feat: add webjs ci, a local CI runner the scaffold workflow reuses - #1472
Merged
Conversation
Local CI (#1471) needs a step list an app declares once and every tool can read without importing app code, so it lives in the package.json `webjs` block next to the #550 dev/start orchestration. This lands the key in the three-surface lockstep (the JSON Schema with recursive definitions for a step, the WebjsConfig type, KNOWN_KEYS plus a nested guard for the recursive shape) and the pure CLI reader that normalizes the shorthand and reports every malformed entry with its JSON path. The reader validates the shapes itself because the boot validator only checks top-level key membership and never follows a $ref, and a step that is silently dropped is a check that never ran, the exact false green local CI exists to prevent. A group nested inside a parallel group takes one slot and runs sequentially, so a `parallel` on it is reported rather than honoured, the same rule Rails' runner applies. The runner and the `ci` command follow in the next commits.
The runner behind the ci command (#1471), modeled on Rails 8.1's ActiveSupport::ContinuousIntegration: a heading and a timed result line per step, a failure list plus one total line, fail-fast, and parallel groups whose steps run on N slots with output captured and replayed whole so nothing interleaves, a nested group taking one slot. Two spawn shapes on purpose. A sequential step inherits stdio and is not detached, so it owns the terminal and Ctrl-C reaches it natively, the split webjs dev already makes for before-steps versus watchers. A captured step is detached (its own process group, reaped on interrupt), has stdin ignored so a TTY-reading tool cannot stop on SIGTTIN and hang the pool, and resolves on close rather than exit, with a bounded grace so a leaked grandchild holding the pipe cannot hang the run. FORCE_COLOR reaches captured children only when the parent's stdout is a TTY; Node has no PTY without a native dependency. Under GitHub Actions each step is folded into a log group and a failure is annotated, so one cloud job running the whole list still names the layer that broke, which is what the per-layer jobs used to buy. Pure of process.exit, console, and the clock, so the slot cap, the fail-fast cutoff, replay atomicity, exit-then-data ordering, and the grace path are all proven with a scripted fake child.
The local CI command (#1471): run the step list package.json declares under webjs.ci, with -f/--fail-fast, --only <title>, --json, and an opt-in --signoff that posts a green commit status through gh signoff after a green run, the Rails 8.1 bin/ci posture. The predicate is the config, not an app/ directory, unlike webjs check, because a workspace root is a legitimate target and this monorepo declares its own list. Nothing declared is exit 1 rather than 0, since a run of zero steps would read as green; the refusal names the workspace members that do declare one. A malformed block refuses with every problem's JSON path and runs nothing. Under --json stdout carries exactly one document and the human report moves to stderr; the exit code is set through exitCode so a non-TTY stdout is never truncated. .env is loaded before the steps, like dev and start, so a local db migrate step sees DATABASE_URL while a CI runner's explicit env still wins. The prose hook's CLI subcommand list gains ci so its drift test keeps passing, and the end-to-end CLI test and the Node + Bun proof script cover the exit codes, the output shape, the env every child sees, fail-fast, --only, the refusals, the GitHub Actions log groups and step summary, and a real interrupt reaping a detached sleep.
Every new app now declares its CI once, in package.json under webjs.ci (#1471): a Setup step, then a Checks group running two at a time (webjs check, webjs doctor, webjs typecheck, a dependency audit) with a sequential Tests sub-group (the server, browser, and e2e layers), plus a `ci` script. The Tests group stays sequential because the server and e2e layers share one SQLite file. Every step is a bare `webjs ...` command, the same bar the before-steps meet, and a Bun app audits with bun audit. The generated GitHub workflow collapses from four hand-restated jobs to one job that prepares the runner and runs `npm run ci`, so the cloud and local lists cannot drift, which is the Rails guide's rule for every provider. The cloud practices stay: a read-only token, a bounded job, a concurrency cancel, and, through the runner's Actions mode, a log group per step, an annotation per failure, and a step table in the job summary, so a failure still names its layer. A team wanting per-layer required checks runs `webjs ci --only` in a matrix, as the workflow comment says. `webjs create --skip-ci` omits the workflow and nothing else (rails new parity); the local list always ships. The pre-commit hook is unchanged per #174: `npm run ci` is the pre-push gate, and the scaffold's agent rule files, playbooks, and PR template now say so.
The framework dogfoods its own local CI (#1471). The root package.json declares a webjs.ci list mirroring the GitHub jobs (setup, the per-app check and doctor plus the two source invariants three at a time, the root test suite, the in-repo app typechecks and suites, the browser suite, the blog e2e, the Bun matrix) behind `npm run ci`, and gallery, examples/blog, and website each declare a shorter list behind their own `ci` script. The root script and every root step invoke this checkout's CLI by path rather than a hoisted bin, because in a linked worktree node_modules/.bin resolves into the primary checkout. The app lists go through their npm scripts because the website's pretest hook copies the ui registry, which a bare test invocation would skip. A repo-health test parses all four blocks through the same reader the command uses and checks that every npm script a step names exists in the package it targets, so a stale step fails here rather than at run time with a message that never mentions the block. The GitHub workflow itself is unchanged: its jobs are the required merge checks.
The ci command and the webjs.ci config key (#1471) on each surface the doc-sync map names: the AGENTS.md CLI reference, config-block bullet, and code-workflow items; the agent skill (a Local CI section in built-ins.md, a one-command section in testing.md, a Testing Defaults bullet in SKILL.md); the docs site (a webjs ci entry under CLI Options on the configuration page with the step shapes, the flags, the Actions surfaces, and the signoff merge gate, a webjs ci command section on the testing page, a deployment-checklist item, and the known-key count); the root and CLI READMEs; and framework-dev.md for the monorepo's own list.
Collaborator
Author
|
Decisions carried from the plan, and two that moved during the build Settled in #1470 and kept: JSON in the Two calls moved while building.
The per-layer status checks the four-job workflow used to provide are replaced, not dropped. The runner detects Two environmental findings worth knowing when reading a local run. A shell exporting |
vivek7405
marked this pull request as ready for review
September 10, 2026 13:06
vivek7405
commented
Sep 10, 2026
vivek7405
left a comment
vivek7405
left a comment
Collaborator
Author
There was a problem hiding this comment.
Read the whole diff (runner, reader, bin, schema/type, scaffold, workflow, tests, docs). The design holds together well: the reader refusing on any shape problem, the inherit-vs-capture split, the resolve-on-close rule, and the scaffold workflow collapsing onto npm run ci are all the right calls, and the tests pin the parts that matter.
Eight findings, two of which I would fix before merging:
- A truncated captured step hangs the process (
ci-runner.js, the grace timer). The result resolves, the summary prints, and thenwebjs cisits there, because the leaked grandchild still holds the pipe and the bin exits throughexitCode. I probed it withsleep 6 & echo hi; exit 0undercloseGraceMs: 300: the result came back at 310ms flagged truncated, the process exited at 6.0s. A leaked dev server never releases the pipe, so that run never exits. - Three surfaces disagree about a nested
parallel. The reader honoursparallelon a group nested inside a SEQUENTIAL group (and acceptsparallel: 1anywhere), while the schema, theWebjsCiNestedGrouptype, the@ts-expect-errorfixture and the docs all say a nested group can never declare it. An editor red-flags a config the CLI runs. Details inline.
The rest: the nested Tests group inside the pool bypasses the progress line bookkeeping (the scaffold default hits it on every TTY run), a second Ctrl-C is swallowed, the total line can never read interrupted, --only with no value silently runs everything, NO_COLOR is not honoured on a TTY, and one stale count on the configuration docs page.
Not flagged but worth knowing: npm audit needs a lockfile and the network, so a --no-install app or an offline machine reds the default list at the audit step. That matches bundle audit in Rails' list, so I left it.
Eight findings from the review on #1472, each with its counterfactual. A captured step whose grandchild kept the pipe open resolved as truncated but left the group alive and the pipe handles open, so the bin printed its summary and then sat until the grandchild died. The grace bound now reaps the group (the child is detached, so it is still addressable) and destroys both pipes; the Node and Bun proof runs a `sleep & echo; exit 0` step and asserts the run settles through the grace and pgrep finds nothing. The reader honoured `parallel` on a group nested inside a sequential group while the schema, the type, and the docs all said a nested group never declares it. One rule now: a nested group refuses `parallel` whatever its parent, and `steps` is required on the schema, the type, and the reader alike (a block with no steps is refused, since zero steps would read as green). A nested group inside a pool bypassed the progress bookkeeping, so on a TTY its replays did not clear the line and no line showed while it ran, which is the scaffold's default shape (Tests inside Checks). Every step in a slot now shares one path. A second Ctrl-C exits with 130 outright, the way Rails' bin/ci lets a repeated interrupt through, so a captured child that ignores SIGTERM cannot trap the user. The total line reads "interrupted" after an interrupt instead of "failed" (the old expression was dead). A bare `--only`, or one followed by a flag, is refused instead of running the whole list. Colour follows NO_COLOR on a TTY, for the runner's own lines and for the FORCE_COLOR handed to captured children. The configuration page's free-form key list names `ci` and counts ten.
The apps job became "In-repo app tests (website + blog + gallery)" in #1371, but scripts/protect-main.sh, and the protection it had applied, still required the old name, so that required check could never report and every PR since has shown as blocked on a check that no longer exists. Name the real job so a green run satisfies the gate.
`npm create webjs` and `bun create webjs` run this wrapper, so the flag `webjs create` gained in #1471 has to reach scaffoldApp from here too, or `npm create webjs my-app -- --skip-ci` would silently scaffold the workflow, the same gap --db once had. The usage and README name it, and a source-level test pins the forwarding, because the wrapper resolves the CLI by bare specifier and a linked worktree would run the primary checkout's copy.
The root webjs.ci list ran its big steps one after another, so a full run took 9m22s on a 24-core machine although no single step needed more than 4m30s. After Setup, everything now sits in one Gate group with three slots, longest first (the blog e2e, the Bun matrix, the browser suite, then the root test suite, the in-repo app suites, and the Conventions checks as slots free up), so the wall-clock is the longest step rather than the sum. Each slot's output is captured and replayed whole, so nothing interleaves. Three rather than six: six overloaded the box, and web-test-runner's Firefox could not launch a test page inside its 30s start timeout. With the e2e alone as the floor, three slots cost no wall-clock. The Conventions group loses its own three-way parallelism, which cost nothing measurable (eight sub-second steps). The repo-health guard still parses the block clean, and framework-dev.md describes the shape.
This was referenced Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1471
webjs ciruns an app's declared step list locally with the Rails 8.1bin/ciexperience: a timed result line per step, a failure list and one total line,--fail-fast,--only <title>,--jsonfor an agent loop, and an opt-in--signoffthat posts a green commit status throughgh signoff. The list lives inpackage.jsonunderwebjs.ci(a string, a{ title, run, env }command, or a{ title, steps, parallel }group whose steps run N at a time with each step's output replayed whole). Every scaffolded app ships a default list plus aciscript, the scaffold's GitHub workflow collapses to one job that runs the same list throughnpm run ciso the two cannot drift,webjs create --skip-ciomits that workflow, and the monorepo root and its three in-repo apps declare their own lists.Research record: #1470.
What changed
webjs.ciin the schema (draft-07definitionsfor the recursive step shape, a nested group cannot declareparallel), theWebjsConfigtype,KNOWN_KEYSplus a nested guard, and the type fixture. The pure readerpackages/cli/lib/ci-config.jsnormalizes the shorthand and reports every malformed entry by JSON path, because the boot validator never follows a$refand a silently dropped step is a check that never ran.packages/cli/lib/ci-runner.js. Sequential steps inherit stdio and are not detached (Ctrl-C reaches them natively). Captured steps are detached with stdin ignored, buffered in arrival order, resolved onclosewith a bounded grace afterexit.FORCE_COLORreaches captured children only when the parent stdout is a TTY. Under GitHub Actions each step is a::group::, a failure an::error::annotation, and a step table lands in$GITHUB_STEP_SUMMARY, so one job still names the layer that broke. Pure ofprocess.exit,console, and the clock.webjs ci. The predicate is the config, not anapp/directory (unlikewebjs check), so a workspace root is a legitimate target. Nothing declared is exit 1 naming the members that declare one; a malformed block is exit 1 with every problem;--jsonputs one document on stdout and the human report on stderr; the exit code goes throughexitCodeso a non-TTY stdout is never truncated.Checksgroup two at a time: check, doctor, typecheck,npm audit --audit-level=highorbun auditon Bun, and a sequentialTestssub-group of server, browser, e2e), aciscript, a one-job workflow with a read-only token, a timeout, and the concurrency cancel,--skip-ci, and the agent rule files, playbooks, PR template, and pre-commit comment now namenpm run cias the pre-push gate. The pre-commit hook itself is unchanged (Lighten pre-commit to convention-check only; move test gate to CI #174).npm run ci(the jobs themselves stay the required checks),gallery/examples/blog/websiteeach declare a shorter one, andtest/repo-health/in-repo-ci-blocks.test.mjspins all four.Deliberately excluded
webjs.ci,gh signoff failposting, MCP exposure.npm run ci(they are the required merge checks).node --testunder a pipe reports as TAP, documented.Test plan
packages/cli/test/ci-config(7),packages/cli/test/ci-runner(12, scripted fake child, fake clock, fake timers, pool cap, replay atomicity, exit-then-data, grace, interrupt, Actions mode),packages/server/test/config(schema lockstep + nested guard),test/types/webjs-config.test-d.tswith three@ts-expect-errorcounterfactuals (type-checked against the branch copy of core via apathsmap; the in-tree run resolves the primary checkout through the linked node_modules and is the known worktree artifact)test/cli/ci.test.mjs(9: refusals, green, red, fail-fast,--json,--only, Actions groups + step summary, help)test/bun/ci-runner.mjson Node and on Bun 1.3.14 (real children, captured stderr, a detachedsleepreaped on interrupt); the Bun matrix filtered toci-runnerandwebjs-configreports no genuine failureshelp,check-target, published-package descriptions (96 pass)test/scaffolds/*(61 + the--skip-cicounterfactual for the library option and the CLI flag),runtime-rewrite(npm run cibecomesbun run ci,npm cistill becomesbun install); a full-stack app generated WITH install runsnpm run ciend to end green (Setup, Conventions, Health, Types, audit, server, browser, e2e) once Playwright Chromium is present, which is the workflow's own install stepnpm run ci -- --only Conventionsgreen (8 steps, three at a time, 2.8s);gallery,examples/blog,websiteeach green on their own list through the branch CLI (the website run after the docs edits, which covers the docs-links and llms guards)npm test, full Node suite. Eight failures, all accounted for: the five that always fail in a linked worktree (listener, listener-overhead, three elision assertions), the config type fixture (bare specifier resolving into the primary, verified separately as above), and two#1451no-any guards that parse tsc's plain output while this shell exportsFORCE_COLOR=3; that file passes 4 of 4 with the variable unsetTwo things a reviewer will notice in a generated app before the next release: the published
@webjsdev/serverschema does not know thecikey yet, so a scaffolded app booted against the published packages prints one boot warning (unknown key "ci") that disappears on release, andnpm run ciin that app must be run through this branch's CLI until the CLI ships.Definition of done
AGENTS.mdcreate --skip-ci, thewebjsblock bullet, code-workflow items 1 and 4).agents/skills/webjs/references/built-ins.mdLocal CI section,references/testing.mdone-command section,SKILL.mdTesting Defaults)packages/cli/AGENTS.md,packages/server/AGENTS.mdwebsite/app/docs/configurationCLI Options entry + key count,testingsection,deploymentchecklist)README.md,packages/cli/README.mdframework-dev.mdworkflow.md, both playbooks, PR template) pluscreate.jsand the CLI helpwebsite/feat:titleexamples/blog/CONVENTIONS.mdReview round
The review on this PR posted eight findings; all eight are fixed in d4e2780 with a counterfactual each (the grace-path reap proven on Node and Bun with a leaked background sleep, the nested-
parallelrule aligned across the reader, schema, type, and docs withstepsrequired on all three, nested-group progress bookkeeping, a second Ctrl-C that exits, the interrupted total line, a refused bare--only,NO_COLORhonoured on a TTY, the docs key count), answered in their threads, and the threads resolved so the required conversation-resolution gate is satisfied; reopen any that needs more.Two follow-on commits while making the PR mergeable: 879e140 points
scripts/protect-main.shat the renamedIn-repo app tests (website + blog + gallery)job (the required check had kept the old name since #1371, so it could never report) and the corrected protection is applied; 0db2086 forwards--skip-cithrough thecreate-webjswrapper sonpm create webjsandbun create webjsmatchwebjs create.Monorepo local CI, measured on a 24-core machine
Sequential root list: 9m22s. Regrouped so the big steps share one
Gategroup: six slots finished in 4m42s but overloaded the box (Firefox could not launch a web-test-runner page inside its 30s start timeout); three slots finished in 4m35s with the browser suite green in 53s, so three is what ships. The wall-clock is now the blog e2e alone (4m30s). Per step this machine beats the Actions runner two to three times (unit 42s vs 2m23s, browser 53s vs 2m41s, app suites 28s vs 1m15s); the only reds in a linked worktree are the known listener and type-fixture artifacts that pass in CI.